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ERRATA  TO  BORC  MRC  REPORT  #85-1 


Bock,  R.  Darrell,  Gibbons,  Robert,  Muraki,  Eiji 

Fall- Information  item  Factor  Analysis 

Lines  were  lost  in  the  first  paragraph  on  Page  23.  The  second 
sentence  should  read: 

"The  effect  of  this  attenuation  is  to  increase  the  rank  of  the 
correlation  matrix,  and  thus  to  introduce  spurious  factors  in  much 
the  same  way  as  variation  in  item  difficulty  introduces  such  factors 
in  the  analysis  of  item  phi-coef ficients 
Also  correct  the  last  phrase  in  the  last  sentence  in  paragraph  2  on 
Page  31  to  read: 

"suggest  the  desirability  of  scoring  separately  the  physicial  science 
and  biological  science  content  of  the  General  Science  test." 


[Abstract] 


A  method  of  item  factor  analysis  based  on  Thurstone's 
multiple  factor  model  and  implemented  by  marginal  maximum 
likelihood  estimation  and  the  EM  algorithm  is  described. 
Statistical  significance  of  successive  factors  added  to 
the  model  is  tested  by  the  likelihood  ratio  criterion. 
Provisions  for  effects  of  guessing  on  multiple  choice 
items,  and  for  omitted  and  not  peached  items,  are 
included.  Bayes  constraints  on /the  factor  loadings  are 
found  to  be  necessary  to  suppress  Heywood  cases. 

Numerous  applications  to  simulated  and  real  data  are 
presented  to  substantiate  the  accuracy  and  practical 
utility  of  the  method.  Analysis  of  the  power  tests  of 
the  Armed  Services  Vocational^Battery  shows  statistically 
significant  departures  from  unidimensionality  J ~ 
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Strictly  speaking,  any  test  reported  in  a  single  score  should 
consist  of  items  drawn  from  a  one-dimensional  universe.  Only 
then  is  it  a  matter  of  indifference  which  items  are  presented  to 
the  examinee.  This  interchangeability  of  items  is  especially 
important  in  adaptive  testing,  where  different  examinees  confront 
different  items. 

Of  the  various  methods  that  have  been  proposed  for 
investigating  the  dimensionality  of  item  sets,  the  most  sensitive 
and  informative  is  item  factor  analysis.  It  alone  is  capable  of 
analyzing  relatively  large  numbers  of  items  jointly  and 
symmetrically,  and  of  assigning  items  to  particular  dimensions 
when  multiple  factors  are  found.  It  can  also  reveal  common 
patterns  of  item  content  and  format  that  may  have  interesting 
cognitive  interpretation. 

Past  methods  of  item  factor  analysis  have,  however,  not  been 
entirely  satisfactory  technically.  Although  conventional 
multiple  factor  analysis  of  the  matrix  of  phi  coefficients  is 
straightforward  computationally,  it  is  well  known  to  introduce 
spurious  factors  when  the  item  difficulties  are  not  uniform. 

This  problem  is  alleviated  by  using  tetrachoric  correlations  in 
place  of  phi  coefficients,  but  this  strategy  also  encounters 
difficulties.  The  matrix  of  sample  tetrachoric  correlation 
coefficients  is  almost  never  positive  definite,  so  the  common 
factor  model  does  not  strictly  apply.  Although  present  methods 
of  calculating  the  tetrachoric  coefficents  are  fast  and  generally 
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accurate  (Divgi,  1979),  they  become  unstable  as  the  values 
approach  +1  or  -1.  When  an  observed  frequency  in  the  four-fold 
table  for  a  pair  of  items  is  zero,  the  absolute  value  of  an 
element  in  the  item  correlation  matrix  becomes  1,  thus  producing 
a  Heywood  case.  These  problems  are  exacerbated  when  the 
coefficients  are  corrected  for  guessing  (Carroll,  1945). 

The  limitations  of  the  item  factor  analysis  based  on 

tetrachoric  correlation  coefficients  have  been  overcome  to  a 

considerable  extent  by  the  generalized  least  squares  (GLS) 

/ 

method  ( Cristof f ersson,  1975;  Muthen,  1978).  Because  this 
method  allows  for  the  large  sample  variance  of  the  estimated 
coefficients,  instabilities  at  the  extremes  are  less  of  a  problem. 
The  GLS  method  requires,  however,  the  generating  and  inverting  of 
the  asymptotic  covariance  matrix  of  the  estimated  tetrachoric 
coefficients;  it  thus  becomes  extremely  heavy  computationally  as 
the  number  of  items  increases.  At  present,  its  practical  upper 

i 

limit  is  about  20  items  (Muthen,  1984). 

It  is  of  some  interest,  therefore,  that  Bock  and  Aitkin 
(1981)  introduced  a  method  of  item  factor  analysis,  based 
directly  on  item  response  theory,  that  is  not  strongly  limited  by 
the  number  of  items.  Although  the  computations  in  their  method 
increase  exponentially  with  the  number  of  factors,  they  increase 
only  linearly  with  the  number  of  items.  The  practical  limit  of 
the  number  of  factors  is  five,  which  is  sufficient  for  most  item 
analysis  applications,  while  as  many  as  60  items  is  not 
excessive . 

Because  the  Bock-Aitkin  approach  uses  as  data  all  distinct 
item  response  vectors,  it  is  called  "full-information"  item 
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factor  analysis  (Bartholomew,  1980),  as  opposed  to  the  limited 

/ 

information  methods  of  Cristof f ersson  and  Muthen  based  on  low- 
order  joint  occurrence  frequencies  of  the  item  scores.  The 
purpose  of  the  present  paper  is  to  present  in  more  detail  the 
derivation  of  the  full-information  factor  analysis,  discuss 
technical  problems  of  its  implementation,  and  describe  our 
experience  with  the  method  in  a  number  of  simulated  and  real  data 
sets . 


1  Derivation  and  statistical  methods 
Bock  and  Aitkin  (1981)  apply  Thurstone's  multiple  factor 
model  to  item  response  data  by  assuming  that  the  m-factor  model. 
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describes  not  a  manifest  variable  j,  but  an  unobservable 

"response  process"  that  yields  a  correct  response  of  person  i  to 

item  j  when  y^  equals  or  exceeds  a  threshold,  y  y  On  the 

assumption  that  u..  is  an  unobservable  random  variable 

2 

distributed  N(0,«r  .),  the  probability  of  an  item  score,  x.  .=  1, 
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Similarly,  the  conditional  probability  of  the  item  score 
x^  =  0,  indicating  an  incorrect  response,  is  the  complement, 

1  -  +j(6).  In  other  words,  the  conditional  response  probability 
is  given  by  a  normal  ogive  model.  Note  that  (1)  is  a 
"compensatory"  model:  greater  ability  in  one  dimension  makes  up 
for  lesser  ability  in  some  other  dimension.  Nothing  prevents, 
however,  the  methods  discussed  here  from  being  applied  to  an 
"interactive"  model  such  as 

y..  =  «  .  ,0 ,  .  +  a.o0_.  +  «  . ,  _0  ,  .8  „  .  +  .  .  .  +  «.  9  .9  +u. 

riJ  Jl  li  32  2a  j 1 2  li  2 a  3mp  ma  pa  a 

(3) 


1.1  Estimation  of  the  item  thresholds  and  factor  loadings 
Like  maximum  likelihood  factor  analysis  for  measured 

II 

variables  (Joreskog,  1967),  the  Bock-Aitkin  method  of  estimating 
parameters  of  an  item-response  model  assumes  that  the  data  have 
been  obtained  from  a  sample  of  persons  drawn  from  a  population 
with  some  multivariate  distribution  of  ability.  Provisionally, 
we  will  assume  that  the  distribution  is  0  ~  N(0,I),  but  this 
assumption  can  be  relaxed  to  allow  for  correlated  factors  and 
non-normal  distributions.  We  also  adopt  the  convention  of  factor 
analysis  that  y  is  distributed  with  mean  zero  and  variance  one, 
so  that 
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On  these  assumptions,  the  marginal  probability  of  the  binary 


response  pattern  is  given  by  the  multiple  integral, 


Pjf  =  P(x  -  *j> 
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Numerical  approximations  of  these  integrals  may  be  obtained 
by  the  m-fold  Gauss-Hermite  quadrature, 


Q  Q  Q 
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(6) 


where  Xk  is  a  quadrature  point  in  m  dimensional  space  and  the 
corresponding  weight  is  the  product  of  weights  for  quadrature  in 
the  separate  dimensions  as  shown.  Equation  (6)  applies,  of 
course,  only  to  uncorrelated  factors.  It  is  an  example  of  a  so- 
called  "product  formula"  for  numerical  integration  and  has  the 
disadvantage  that  the  number  of  terms  in  the  sum  is  an 
exponential  function  of  the  number  of  dimensions.  Fortunately, 
the  number  of  points  in  each  dimension  can  be  reduced  as  the 
dimensionality  is  increased  without  imparing  the  accuracy  of  the 
approximations.  Thus,  factor  analysis  with  five  factors  can  be 
performed  with  good  accuracy  with  few  as  three  points  per 
dimension.  In  that  case,  3^  =  243  quadrature  points  are  required, 
and  the  solution  is  accessible  with  a  fast  computer. 
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Given  the  frequencies,  r^  of  the  response  patterns,  x^  for  n 
items  and  a  sample  of  N  persons,  the  number  of  distinct  pattern 
is  s  ^  max(2n,  N) ,  and  the  probability  of  the  sample  is 


Lm  =  P(X)  = 


r  1  r  1  . 
12 
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Then  the  maximum  likelihood  estimates  of  the  threshold  and 
factor  loadings  are  those  values  that  maximize  (7).  To  simplify 
the  expression  of  the  likelihood  equations,  it  is  convenient  to 
write 


Y  .  -  I  Of  ..  0  .  . 

3  k=1  Jk  k! 


=  c  .  -  E  a  0 ,  . 
3  k-i  Jk  kl 


that  is,  to  express  the  model  in  terms  of  the  intercept  and 
slopes  of  the  response  function.  From  MML  estimates  of  the 
latter,  MML  estimates  of  the  former  may  be  obtained  by 


T  •  =  -  c  .  /  d  . 
J  J  J 


of  ..  =  a  ..  /d  . 
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where 
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Three  analyses  were  performed:  1)  no  guessing  assumed  in  the 
data  or  in  the  analysis;  2)  guessing  in  the  data  but  no  guessing 
assumed  in  the  analysis;  3)  guessing  assumed  in  the  data  and  in 
the  analysis.  In  all  of  these  analyses,  the  item  intercepts  and 
factor  loadings  were  estimated  from  the  data  by  an  EM  marginal 
maximum  likelihood  solution  in  which  the  iterations  began  from 
the  principal  factors  of  the  sample  tetrachoric  correlation 
matrix  (with  communality  iteration).  Item  guessing  paramaters, 
on  the  other  hand,  were  set  at  their  assumed  values  and  not 
estimated . 

It  is  instructive  to  examine  the  effects  of  guessing  and  the 
effect  of  correction  for  guessing  on  the  item  facilities  and  the 
item  tetrachoric  correlations.  These  relationships  are  shown 
graphically  in  Figures  3-1  and  3-2.  Figure  3-1  confirms  the 
well-known  effect  of  guessing  on  item  facilities.  Deviation  of 
the  observed  facilties  from  their  theoretical  values  as  a 
function  of  the  true  item  intercepts  from  their  theoretical 
values  is  due  entirely  to  sampling. 


***************************** 

INSERT  FIGURES  3-1  &  3-2  HERE 
***************************** 


Figure  3-2  shows  the  average  tetrachoric  correlations  for 
sets  of  three  successive  items  ordered  by  facility.  When 
guessing  is  not  assumed  or  corrected  for,  the  average 
coefficients  are  near  their  theoretical  value  of  .5  at  all  levels 


of  facility.  When  guessing  is  present,  but  uncorrected,  the 


Some  idea  of  the  overall  speed  of  the  present  implementation 

is  given  by  the  IBM  3081  cpu  time  for  the  test  of  general  science 

discussed  in  section  4.3.  The  total  go-step  cpu  time  for  a  three 

3 

factor  solution  with  25  items,  1,178  subjects,  3  =27  quadrature 

points,  35  EM  cycles,  a  maximum  of  five  M-step  iterations,  and 
numbers  of  omits  as  shown  in  Table  4-3,  was  11  minutes  and  43 
seconds . 

3  Simulation  Studies 

As  a  check  on  both  the  derivation  and  the  computer 
implementation,  we  performed  the  following  analyses  of  simulated 
data . 

3.1  A  one-factor  test 

This  simulation  demonstrates  the  capacity  of  marginal  maximum 
likelihood  factor  analysis  to  identify  unidimensional  item  sets 
in  the  presence  of  guessing.  To  verify  that  the  analysis  has  no 
tendency  to  produce  difficulty  factors,  the  item  facilities  were 
chosen  to  span  a  range  larger  than  is  typical  of  most  tests  of 
ability.  This  was  done  by  setting  the  item  intercepts  and 
equally  spaced  points  between  -2.0  and  +2.0.  All  item  slopes 
were  set  at  1.0,  corresponding  to  a  factor  loading  of  .707,  and 
all  guessing  parameters  (lower  asymptotes)  were  set  at  0.25. 
Responses  with  and  without  guessing  were  simulated  for  1000 
subjects  drawn  randomly  from  a  normal  (0,1)  distribution  of 
ability. 
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is  bounded  between  0  and  1,  the  beta  prior 


f  (» j2 )  -  B(p.q)  1  (cr..2)P  V  -  1 


(25) 


with  q  =  1  be  used  to  hold  c r  away  from  zero  without  restricting 
its  approach  to  1.  When  m  =  2,  for  example,  MAP  estimation  with 
this  prior  adds  the  penalty  function, 
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to  the  likelihood  equations,  and  adds  the  ridge, 
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to  the  information  matrix  of  the  M-step  maximum  likelihood 
estimator.  Muraki  (1984)  finds  that  this  approach  performs  well 
in  full-information  item  factor  analysis. 

2 . 5  Computing  times 

Computing  times  depend  upon  the  number  of  factors,  items, 

.  subjects,  quadrature  points,  EM  cycles,  M-step  iterations,  and 
the  proportion  of  omitted  or  not  presented  items.  The  preliminary 
steps  of  data  input  and  computing  starting  values  are  not  very 
time  consuming  relative  to  the  full-information  solution.  Most 
of  the  time  in  the  latter  is  accounted  for  by  the  evaluation  of 
likelihoods  in  the  E-step;  the  M-step  times  are  relatively  small. 
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To  be  analyzed  by  the  MINRES  method  (Harman,  1976),  the 
tetrachoric  matrix  must  be  positive  definite.  The  corrected 
matrix  obtained  through  the  centroid  method,  on  the  other  hand, 
may  have  zero  and  negative  roots.  Therefore,  a  preliminary 
"smoothing"  of  the  tetrachoric  correlation  coefficient  matrix  is 
needed  before  the  principal  factor  analysis  is  carried  out.  The 
smoothed  tetrachoric  correlation  matrix  is  produced  from  the 
eigenvectors  associated  with  the  positive  roots,  after  renorming 
the  sum  of  the  roots  to  equal  the  number  of  items.  The 
reproduced  positive  definite  tetrachoric  correlation  matrix  is 
then  analyzed  by  the  MINRES  method  to  obtain  good  starting  values 
for  the  full-information  factor  analysis. 

2.4  Constraints  on  Item  Parameter  Estimates 

An  undesirable  feature  of  maximum  likelihood  factor  analysis 
is  its  tendency  to  produce  Heywood  cases,  i.e.,  boundary 
solutions  in  which  the  uniqueness  is  zero  for  one  or  more 
variables.  These  cases  also  occur  in  full-information  item 
factor  analysis,  the  symptom  being  one  or  more  continually 
increasing  item  slopes  as  the  EM  cycles  continue. 

One  way  of  handling  this  problem  is  to  assume  a  restricted 
prior  distribution  on  some  of  the  item  parameters  and  to  employ 
maximum  a  posteriori  (MAP)  estimation  to  maximize  the  posterior 
probability  density  of  the  parameters  rather  than  the  likelihood. 
Martin  and  McDonald  (1973)  assume  an  exponential  distribution  for 
the  uniqueness  and  Lee  (1981)  employs  an  inverted  gamma  prior  for 
this  purpose.  Mislevy  (1984)  suggests  that,  since  the  uniqueness 
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Marginal  frequencies  are  computed  by 


n  1. 

=  nl.  + 

pin: 

n'o. 

=  no.  + 

CliH; 

n’.l 

=  n.l  + 

qjn 

and 

n'.o 

=  n.o  + 

qjn 

Therefore , 

n '  . 

+  n  ' 

=  n ' 

1 . 

0 . 

because 

.Pi  + 

qi  =  PJ 

+  q 

2.3  Preliminary  Smoothing  of  the  Tetrachoric  Correlation  Matrix 
Although  the  correction  for  omits  makes  the  calculation  of 
most  of  the  tetrachoric  correlations  possible,  there  are  still 
occasional  instances  in  large  matrices  where  a  value  close  to  0 
appears  in  the  minor  diagonal  of  the  tables  of  a  few  item 
pairwise  joint  frequencies.  Since  no  admissible  coefficient  can 
be  computed  from  such  a  table,  some  method  of  imputing  a  value  is 
required.  A  reasonable  approach  is  to  assume  that  the  matrix  of 
tetrachoric  correlations  is  dominated  by  a  single  factor.  In 
that  case,  Thurstone's  centroid  formula  applied  to  the  valid 
correlations  can  be  used  to  estimate  the  item  factor  loadings 
from  which  the  missing  coefficients  can  be  calculated.  Because 
the  full-information  analysis  uses  the  tetrachoric  correlations 
only  for  starting  values,  no  bias  of  the  solution  results  from 
these  imputations. 
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Let  us  denote  as  the  observed  frequency  in  the  3x3 

table  whose  categories  are  pass,  fail,  and  omit.  Thus,  the 
observed  frequency  table  may  be  expresserd  as  in  Table  2-3. 


INSERT  TABLE  2-3  HERE 


If  the  proportions  of  correct  and  incorrect  responses  based 
on  non-omitted  responses  are  denoted  by  p's  and  q's  respectively, 
they  are  computed  by 

pd  =  (n12  +  n10)/N.. 

qi  =  {n01  +  n00)/N“ 

pj  =  (nll  +  n0l)/N“ 


and 


where 


(n10  +  n00)/N' 


(22 


N"  *  nU  +  n10  +  n01  +  "00 

If  we  can  assume  that  omitted  responses  can  be  reallocated  to 
correct  and  incorrect  responses  proportional  to  p's  and  q's,  the 
following  corrected  frequencies  n'^  are  obtained: 


n 

11 

"  nll 

+ 

p  .n, 

*3  lx 

+ 

p .  n  , 
*1  xl 

+ 

p .  p  .n 
i  3  xx 

n 

10 

nio 

+ 

q  .n, 

^ 3  lx 

+ 

p .  n  . 
xO 

+ 

p .  q  .n 

xx 

• 

n 

01 

n01 

+ 

^ j  Ox 

+ 

q.n  , 
xl 

+ 

q  .  p  .  n 

^  1  J  XX 

and 


n ' 


00 


+  q  ,nn  +  q.n  _  +  q.q.n 
00  Ox  xO  i  J  xx 


I  O  Q 


2.2  Correction  for  Omitted  Responses 

A  disadvantage  with  Carroll's  formula  for  correcting  the 
tetrachoric  is  that  it  fairly  often  produces  a  zero  or  negative 
values  in  an  off-diagonal  element  of  the  four-fold  table.  If  all 
omitted  responses  are  recoded  as  incorrect  responses,  the 
observed  proportions,  n1Q,  nQ1,  and  nQ0,  tend  to  be  inflated. 
Since  the  positive  c  rrected  proportions  are  obtained  only  if 
n00/n0  ^wj  and  n00/n  o^wi'  ne9ative  corrected  proportions  are  the 
likely  result.  This  problem  is  almost  always  encountered  because 
omitted  responses  are  frequently  found  in  cognitive  testing.  A 
possible  solution  for  this  problem  is  to  allocate  omitted 
responses  to  the  categories  of  correct  and  incorrect  responses  as 
shown  below.  This  correction  for  omits  must  be  made  before  the 
correction  for  guessing. 
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The  guessing  parameter  is  the  probability  of  observing  a 
correct  response  when,  given  the  true  state  of  mastery  for  the 
item,  the  response  should  be  failure.  Thus,  the  observed 
proportion  of  passing  is  the  sum  of  the  proportion  of  the  true 
state  of  mastery  and  the  joint  proportions  of  the  corresponding 
guessing  and  the  true  failure  state.  Therefore,  we  obtain 
n„  =  n  '  +  g  .n  ' 

".1  =  "'.1  +  gj"'  .0 

nn  =  "'ll  +  gi"'oi  +  gj"'io  +  gigj"'oo 


"'ll  +  "'oi  +  "'io  +  "'oo  =  1 


From  Equations  (17),  we  solve  the  corrected  proportions  n's 
in  terms  of  the  observed  proportion  n  and  guessing  parameters  g's 
as  follows: 

n'oo  *  n00/wiwj 

"'oi  -  (Wj”oi  -  9j"00)/WiWj 

"'10  ■  ,wi"io  -  sinoo)/wiw3 


"'u  ■  1  -"00  -"01  -"10  (la; 

where  w.  ■  1  -  g.  and  w .  =  1  -  g . . 

To  convert  the  item  statistics  for  chance  success,  we  proceed 
as  follows.  The  conversion  of  the  kth  factor  loading  to  the 

provisional  slope  estimate  a  ..  is 

ajk  =  *jk/erj  ' 


where 


2  2 
'i  *  1  -r  *jk  ■ 
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In  the  full-information  analysis,  a  similar  solution  results  from 
substituting  for  the  normal  ogive  response  function,  the  guessing 
model , 


+,*(0)  =  g,  +  (i-g,)*,(0)  . 

3  3  33  (16) 


* 

where  g^  is  the  lower  asymptote  of  (0 ) .  The  lower  asymptotes 
for  the  items  may  be  estimated  by  marginal  maximum  likelihood 
along  with  the  intercept  and  slope  parameters,  possibly  with  a 
prior  distribution  assumed  for  g^  in  the  M-step. 

If  the  item  response  model  with  guessing  parameter  is  used 
for  the  full-information  factor  analysis,  the  tetrachoric 
correlation  matrix  must  be  corrected  for  guessing  prior  to  the 
principal  factor  analysis  in  order  to  produce  good  starting 
parameter  values.  To  express  Carroll's  correction  method  in 
terms  of  the  proportions  in  the  2x2  table,  let  us  denote  by  g^ 


respectively.  Denote  by  the  observed  proportions  in  the 

original  2x2  table,  which  are  affected  by  chance  success,  and 
by  n'jj  'the  proportions  in  the  corrected  2x2  table,  which 
exclude  chance  success.  Thus,  the  original  and  corrected 
contingency  tables  may  be  expressed  as  in  Tables  2-1  and  2-2, 
respectively . 


**************************** 
•  INSERT  TABLES  2-1  &  2-2  HERE 

> 
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the  computation.  For  the  same  reason,  it  is  important  that  the 
solution  begin  from  accurate  starting  values.  A  good  strategy 
to  obtain  starting  values  is  to  perform  a  principal  factor 
analysis,  with  communality  iteration,  on  the  matrix  of 
tetrachoric  correlations  for  the  items  in  question.  The 
tetrachoric  correlation  matrix  is  corrected  for  guessing,  and  for 
missing  values,  and  is  conditioned  to  be  positive  definite  so 
that  the  principal  factor  analysis  can  produce  good  starting 
values  for  the  full-information  factor  analysis. 

Since  the  factors  of  the  principal  factor  analysis  are 
orthogonal,  their • loadings  are  suitable  for  the  full-information 
solution  after  conversion  to  item  intercepts  and  slopes.  Item 
intercept  and  slope  estimates  based  on  the  full-information 
method  are  then  converted  again  into  factor  loadings.  The 
resulting  full-information  factor  pattern  can  be  rotated 
orthogonally  to  the  varimax  criterion  (Kaiser,  1958).  With  the 
varimax  solution  as  target,  the  pattern  can  be  rotated  obliquely 
by  the  promax  method  (Hendrickson  and  White,  1964).  The  promax 
pattern  is  especially  useful  for  identifying  two-dimensional 
subsets  of  items  into  which  a  larger  set  that  may  be  partitioned  in 
order  to  measure  more  than  one  dimension. 

2.1  Correction  for  Guessing 

Carroll  (1945,  1983)  has  warned  against  artifacts  introduced 
into  item  factor  analysis  by  guessing  on  multiple  choice  items. 

To  suppress  these  effects,  he  proposes  corrections  to  the  four¬ 
fold  tables  from  which  the  tetrachoric  correlations  are  computed. 


of  the  model  relative  to  the  general  multinomial  alternative  is 
G2  -  2  Z  r  ln(r  /NP  )  . 

(15 

where  is  computed  from  the  maximum  likelihood  estimates  of  the 
item  parameters.  The  degrees  of  freedom  are 
2n  -  n(m+l)  +  m(m-l)/2 

In  this  case,  the  goodness  of  fit  test  can  be  carried  out 

after  performing  repeated  full-information  analyses,  adding  one 

2 

factor  at  a  time.  When  G  falls  to  insignificance,  no  further 
factors  are  required. 

When  the  number  of  patterns  is  larger  than  the  sample  size, 
however,  some  of  the  expected  frequencies  may  be  near  zero.  In 
this  case,  (15),  or  other  approximations  to  the  likelihood  ratio 
statistic  for  goodness-of-f it ,  becomes  inaccurate  and  cannot  be 
relied  on.  Haberman  (1977)  has  shown,  however,  that  the 
difference  in  these  statistics  for  alternative  models  is 
distributed  in  large  samples  as  chi-square,  with  degrees  of 
freedom  equal  to  the  difference  of  respective  degrees  of  freedom, 
even  when  the  frequency  table  is  sparse.  Thus,  the  contribution 
of  the  last  factor  added  to  the  model  is  significant  if  the 
corresponding  change  of  chi-square  is  statistically  significant. 
We  investigate  properties  of  the  change  chi-square  statistic 
empirically  in  sections  3  and  4. 

2  Implementation  of  the  Full-Information  Factor  Analysis 
Typically,  EM  solutions  converge  so  slowly  that  devices  such 
as  Ramsay's  (1975)  acceleration  method  must  be  used  to  speed  up 


algorithm  for  marginal  maximum  likelihood  estimation  as  given  by 

Dempster,  Laird,  and  Rubin  (1977).  Equations  (13)  and  (14)  comprises 

the  E-step,  in  which  expectations  of  "complete  data"  statistics  are 

computed  conditional  on  the  "incomplete  data."  Equation  (12)  is 

the  M-step,-  in  which  conventional  maximum  likelihood  estimation 

is  carried  out  using  the  expectations  in  place  of  complete  data 

statistics.  Because  the  expectations  depend  upon  the  parameters 

to  be  estimated,  however,  the  calculations  must  be  carried  out 

iteratively.  Given  starting  values  for  the  parameters,  a  Qm 

table  of  expected  frequencies,  r.  ,  of  numbers  of 

^ ' qiq2 ‘  •  • 

correct  responses  at  each  point,  X^,  is  built  up  for  each  item  by 

distributing  corresponding  item  score  weighted  by  the  posterior 

probability  of  the  response  pattern,  x^ ,  occuring  at  point  X^. 

Similarly,  N  is  obtained  as  the  sum  of  the  weights  for 

qlq2 ' * ' qm 

each  point.  From  these  statistics,  improved  estimates  of  the 
item  parameters  are  obtained  in  the  M-step  by  applying  the 
appropriate  maximum  likelihood  solution  to  the  table 
corresponding  to  the  item  in  question.  In  the  present  case,  any 
standard  procedure  for  multiple  probit  analysis  will  suffice  for 
the  M-step.  But  the  procedure  is  general  for  any  item  response 
model;  if  a  logistic  response  model  were  assumed,  a  multiple 
logit  analysis  would  appear  in  the  M-step. 

1.2  Testing  the  number  of  factors 

If  the  sample  size  is  sufficiently  large  that  all  2n  possible 
response  patterns  have  expected  values  greater  than  one  or  two, 
the  chi-square  approximation  for  the  likelihood  ratio  test  of  fit 
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The  multiple  integral  in  this  equation  may  be  evaluated 
numerically  by  repeated  Gauss-Hermite  quadrature  as  follows: 
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The  pseudo- frequency  r .  is  an  entry  in  a  Qm 

J'qiq2' ' .qm 

dimensional  array  in  which  each  cell  corresponds  to  an  m-tuple  of 
quadrature  points  for  a  given  item.  The  entries  in  this  table 
are  the  numbers  of  examinees  with  abilities  equal  to  the  vector 
Xj  who  are  expected  to  respond  correctly  to  the  item,  given  the 
sample  data. 

The  quantity  N  is  the  margin  of  this  array  summed 

qlq2’ ’ '  qm 

over  items;  it  is  the  expected  number  of  persons  with  ability  X^ 


and  is  normalized  to  the  sample  size. 

These  equations  correspond  to  the  steps  in  the  so-called  "EM 


Notice  that  the  item  threshold  In  this  model  is  not  an 
Invariant  statistic:  it  depends  upon  the  distribution  of  ability 
in  the  sample.  In  addition,  it  is  on  the  response  process 
dimension  and  not  on  an  ability  dimension.  The  invariant 
location  parameter  of  the  one  dimensional  model  does  not  exist  in 
the  multidimensional  case;  the  value  of  one  ability  that 
corresponds  to  a  given  probability  of  correct  response  is  a 
linear  function  of  the  other  abilities. 

The  likelihood  equation  for  a  general  item  parameter,  Vy  is: 
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average  tetrachoric  coefficients  are  attenuated,  and  the  effect 
becomes  greater  as  the  items  become  harder.  At  the  highest 
levels  of  difficulty,  most  of  the  correct  responses  are  due  to 
chance  successes  and  the  tetrachoric  correlation  is  essentially 
zero.  The  effect  of  this  attenuation  is  to  increase  the  rank  of 
the  correlation  matrix,  and  thus  to  introduce  spurious  factors  in 
the  analysis  of  item  phi  coefficients.  Table  3-1  shows  the 
distinctive  pattern  of  loadings  on  the  spurious  second  factor 
that  results  when  guessing  effects  are  ignored  in  the  analysis: 
items  on  either  extreme  of  the  difficulty  continuum  tend  to  have 
opposite  signs. 

********************* 

INSERT  TABLE  3-1  HERE 
********************* 

When  the  guessing  model  is  assumed,  both  in  calculating  the 
tetrachoric  correlations  and  in  the  response  function  for  the 
marginal  maximum  likelihood  factor  analysis,  the  deleterious 
effects  of  guessing  are  largely  eliminated.  As  shown  in  Table  3-2, 
the  likelihood  ratio  test  for  the  addition  of  a  second  factor, 
which  is  significant  when  the  no-guessing  model  was  applied  to 
guessing  data  in  Analysis  2,  falls  to  insignificance  when 
guessing  is  assumed  in  Analysis  3.  The  estimated  first  factor 
loadings,  which  were  much  attenuated  in  Analysis  2,  are  raised  in 
Analysis  3  to  near  their  true  values. 

********************* 

INSERT  TABLE  3-2  HERE 


These  results  illustrate  the  robustness  of  the  analysis  in 
identifying  the  number  of  factors  and  in  estimating  the  factor 
loadings  in  the  presence  of  a  wide  range  of  item  difficulty  and 
of  guessing  at  a  typical  level  of  chance  success.  This 
relatively  successful  performance  of  the  method  is  qualified, 
however,  by  its  use  of  assigned  rather  than  estimated  guessing 
parameters.  Underestimation  of  these  parameters  would  certainly 
leave  some  effect  of  guessing  in  the  solution  and  possibly 
produce  spurious  factors. 

3.2  A  two-factor  test 

To  demonstrate  the  power  of  MML  item  factor  analysis  to 
detect  a  second  factor,  a  simulation  study  was  conducted  based  on 
an  analysis  of  the  Auto  and  Shop  Information  subtest  of  the  Armed 
Services  Vocational  Aptitude  Battery.  This  subtest  was 
constructed  from  the  previously  separate  Auto  Information  and 
Shop  Information  test  items  of  the  earlier  Army  Classification 
Battery.  As  discussed  in  section  4,  three  factors  were  extracted 
from  the  observed  data  for  1,178  cases  by  a  stepwise  MML  item 
factor  analysis.  As  shown  in  Table  3-3,  the  change  in  the 
likelihood  ratio  chi-square  due  to  inclusion  of  a  second  factor 
was  significant,  but  that  due  to  the  third  factor  was  not. 


The  resulting  estimated  factor  loadings  of  the  two-factor 
solution  are  plotted  in  the  upper  panel  of  Figure  3-3  after 
orthogonal  rotation  to  the  varimax  criterion.  The  axes  after 
oblique  rotation  to  the  promax  criterion  are  also  shown. 

Although  items  3  and  10,  and  possibly  2,  are  misclassif ied ,  the 
plot  clearly  separates  the  auto  and  shop  moieties.  Based  on 
these  loadings  for  the  25  items,  binary  scores  of  1000  simulated 
subjects  were  generated  according  to  the  formula  (16)  with  the 
lower  asymptote  values  shown  for  the  Auto-Shop  test  in  Table  4-5. 
Factor  scores  were  drawn  randomly  from  a  standard  normal 
distribution. 

These  simulated  data  were  then  analyzed  by  the  MML  item 
factor  analysis  with  lower  asymptotes  assigned  the  specified 
values.  Again  two  significant  factors  were  found.  The  lower 
panel  of  Figure  3-3  gives  the  resulting  varimax  rotated  factor 
loadings  and  promax  rotated  axes.  The  MML  estimates  based  on  the 
simulated  responses  are  very  similar  to  their  generating  values. 

********************** 

INSERT  FIGURE  3-3  HERE 


A  Applications 

In  this  section,  the  full-information  analysis  is  applied  to 


a  number  of  empirical  data  sets. 


4.1  Analysis  of  the  LSAT  Section  7  with  and  without  guessing 

Table  4-1  shows  the  tetrachoric  correlations  uncorrected  and 
corrected  for  guessing  assuming  an  asymptote  of  0.2  for  all 
items.  Note  that  the  correction  increases  the  magnitude  of  all 
the  coefficients. 


********************* 

INSERT  TABLE  4-1  HERE 
********************* 

Figure  4-1  shows  the  increase  in  marginal  log  likelihood  in 
successive  EM  cycles  of  a  two  factor  solution  without  guessing. 
Even  with  the  use  of  Ramsay  accelerator,  the  likelihood  increases 
slowly  as  the  solution  point  is  approached.  Twelve  cycles  were 
required  for  convergence. 


********************** 

INSERT  FIGURE  4-1  HERE 
********************** 

With  five  items  and  1000  subjects,  these  data  permit  the 
accurate  calculation  of  goodness-of-f it  chi-square  as  well  as 
change  chi-squares,  as  seen  in  Table  4-2.  Both  give  evidence  of 
a  marginally  significant  second  factor,  and  there  is  no 
indication  that  the  guessing  correction  improves  the  solution. 
Similar  conclusions  are  indicated  by  the  residuals  from  the 


tetrachoric  coefficients  shown  in  Table  4-3. 


*  *  * 


****** 


INSERT  TABLES  4-2  AND  4-3  HERE 


Figure  4-2  shows  the  principal  factor  starting  values  (open 
circles)  and  MML  estimates  of  the  factor  loadings  from  the  non¬ 
guessing  solution  (closed  circles).  It  is  apparent  that  loadings 
on  the  second  factor  are  changed  most  by  the  full-information 
solution,  and  that  the  item  with  the  most  extreme  correlations, 
item  5,  is  most  affected.  The  factor  axes  rotated  to  the  varimax 
and  promax  criteria  show  that  i,tem  2  mostly  clearly  determines 
the  second  factor. 

********************** 

INSERT  FIGURE  4-2  HERE 
********************** 

4.2  The  quality  of  Life 

Campbell,  Converse,  and  Rodgers  (1976)  assessed  13  aspects  of 
the  quality  of  life  in  1800  randomly  selected  respondents  to  a 
NORC  survey.  Respondents  rated  each  quality  in  terms  of  their 
satisfaction  with  that  aspect  of  their  life.  For  present 
purposes,  these  ratings  were  dichotomized  at  the  neutral  category, 
and  a  random  sample  of  1000  cases  was  selected.  A  five  factor 
solution  for  these  data  is  displayed  in  Table  4-4.  Inspection  of 
Table  4-4  clearly  reveals  five  easily  interpretable  dimensions 
underlying  the  quality  of  life;  1)  health,  2)  satisfaction  with 
the  living  environment  (i.e.  neighborhood  and  house  quality),  3) 
satisfaction  with  everyday  life  (i.e.  job,  leisure,  friends, 

27 


family  and  overall  life),  4)  financial  satisfaction  (i.e.  savings 
and  standard  of  living)  and  5)  satisfaction  with  self.  In  terms 
of  level  of  satisfaction  as  indicated  by  the  item  thresholds  in 
Table  4-4,  most  respondents  were  satisfied  with  their  health, 

J 

family  and  friends;  however,  only  the  most  satisfied 
respondents  also  reported  satisfaction  with  their  savings  and 
education . 

********************* 

INSERT  TABLE  4-4  HERE 
********************* 

As  a  further  verification  of  the  factor  solution,  a  limited- 

t 

information  GLS  analysis  was  also  performed  (Muthen,  1978).  The 

/ 

results  of  this  analysis,  employing  Muthen' s  LISCOMP  program,  are 
shown  in  Table  4-4;  they  correspond  closely  to  those  of  the  full- 
information  solution.  Parameter  estimates  are  quite  similar  and 
the  chi-square  statistics  for  the  improvement  of  fit  with  the 
addition  of  each  new  factor  were  virtually  identical.  The 
concordance  between  these  two  computationally  different  methods 
is  taken  as  strong  support  for  the  validity  of  both  the  methods 
and  the  correctness  of  their  implementations. 

4.3  Power  tests  of  the  Armed  Services  Vocational  Aptitude  Battery 
( ASVAB )  Form  8 A 

The  latent  dimensionality  of  each  of  the  eight  power  tests  of 
the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  was 
examined  in  a  ten-percent  random  sample  of  data  from  the  Profile 


of  American  Youth  Study  (see  Bock  and  Moore,  1985).  The  data 
base  from  which  this  sample  was  extracted  consisted  of  ASVAB 
item  responses  of  11,817  members  of  the  Youth  Panel  of  the 
National  Longitudinal  Study  of  Labor  Force  Participation  (NLS). 
The  number  of  cases  in  present  analysis  is  1,178.  The  battery 
was  administered  under  standard  conditions  by  personnel  of  the 
National  Opinion  Research  Center  (NORC).  Because  the  panel  • 
members  were  selected  in  a  clustered  probability  sample,  the 
design  effect  is  greater  than  unity  and,  as  we  point  out  below, 
some  adjustment  of  the  conventional  random  sampling  statistical 
criteria  is  necessary. 

Previous  analysis  of  these  data  by  Bock  and  Mislevy  (1981) 
provides  the  estimates  of  the  lower  asymptote  parameters  for  each 
item  shown  in  Table  4-5.  These  values  were  used  when  the 
guessing  model  was  assumed  in  the  full-information  item  factor 
analyses.  Inasmuch  as  the  examinees  were  given  no  explicit 
instructions  about  guessing  or  omitting  items,  it  seems 
appropriate  to  score  omits  as  incorrect.  Either  because  they  run 
out  of  time  or  find  the  items  too  difficult,  however,  some 
examinees  stop  responding  before  they  complete  all  the  items  on  a 
given  test.  In  these  cases,  we  consider  all  items  following  the 
last  non-omitted  item  to  be  "not  presented".  This  avoids  the 
spurious  association  among  items  later  in  the  test  when  it  is  not 
operating  strictly  as  a  power  test  for  all  examinees.  (See, 
however,  the  special  handling  of  the  Word  Knowledge  test.) 


********************* 

INSERT  TABLE  4-5  HERE 
********************* 
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The  results  of  the  item  factor  analyses,  with  the  estimated 
factor  loadings  shown  both  in  their  principal  factor  and  promax 
rotations  are  shown  in  Tables  4-6  through  4-13.  These  tables 
include  the  change  chi-squares,  degrees  of  freedom,  and 
probability  levels  due  to  inclusion  of  additional  factors.  Also 
shown  are  percents  of  variance  associated  with  each  of  the 
principal  factors  (i.e.,  the  percent  that  the  corresponding 
latent  root  of  the  reproduced  item-correlation  matrix  is  of  the 
trace  of  that  matrix)  and  the  intercorrelations  of  the  promax 
factors . 

****************************** 

INSERT  TABLES  4-6  TO  4-13  HERE 
****************************** 


Except  in  one  instance  discussed  below,  the  factors  found  by 
the  full-information  analysis  to  be  statistically  significant 
corresponded  to  obvious  and  often  cognitively  interesting 
features  of  the  items.  Although  we  cannot  exhibit  actual  items 
from  this  test,  which  is  still  secure,  we  can  convey 
descriptively  the  nature  of  the  factors.  Those  readers  who  have 
access  to  the  test  can  check  our  interpretation  by  examining  the 
items  in  connection  with  the  factor  loadings  in  the  tables.  The 
promax  loadings  are  most  useful  for  this  purpose.  The  number  of 
EM  cycles  was  35  in  each  case. 

General  Science  ( GS )  (Table  4-6).  Even  with  the  guessing 
accounted  for,  a  significant  second  factor  is  found.  The 
corresponding  change  in  chi-square  is  more  than  five  times  its 
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degrees  of  freedom  and  would  remain  significant  with  an  assumed  a 
design  effect  as  large  as  2.0.  The  promax  factors  are  easily 
interpreted.  The  first  is  essentially  physical  science,  and  the 
second  is  biological  science  --or  more  precisely,  health 
science.  These  factors  are  substantially  correlated  (r  =  0.740), 
reflecting  the  large  percent  of  variance  (51.5)  attributable  to 
the  first  principal  factor  in  contrast  with  4.4  percent  for  the 
second . 

The  finding  of  two  factors  in  GS  agrees  with  the  observation 
of  Bock  and  Mislevy  (1981)  that  there  is  an  item-by-sex 
interaction  in  this  test  such  that  male  examinees  tend  to  do 
better  on  physical  science  items  and  female  examinees  better  on 
biological  and  health  science  items.  These  results,  in  addition 
to  the  fact  that  various  civilian  and  military  occupational 
specialties  divide  along  the  same  lines,  suggest  that  the 
desirability  of  scoring  the  physical  science  and  biological 
science  content  of  the  General  Science  test  should  be  scored 
separately . 

Arithmetic  Reasoning  (AR)  (Table  4-7).  There  is  clear 
evidence  for  a  significant  second  factor  in  this  test,  but  not 
for  a  third  factor  if  a  design  effect  of  2.0  is  assumed.  The 
second  factor  makes  a  very  minor  contribution  to  variance, 
however,  and  is  represented  by  only  three  items  with  high  promax 
loadings.  These  items  involve  computation  of  interest,  suggesting 
some  sort  of  business  arithmetic  factor.  Although  additional 
items  might  be  added  to  better  define  such  a  factor,  it  appears 
to  be  of  minor  importance  in  assessing  arithmetic  reasoning 
ability. 
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Word  Knowledge  (WK)  (Table  4-8).  More  strongly  than  other 
tests  in  the  ASVAB ,  Word  Knowledge  appears  in  Form  8  with  its 
items  ordered  from  easy  to  hard  in  difficulty.  It  also  has  a 
relatively  short  time  limit--ll  minutes  for  35  items.  As  a 
consequence  of  these  two  conditions,  the  question  of  how  to 
handle  omitted  responses  at  the  end  of  the  test  is  troublesome. 
Omitted  items  could  mean  either  that  the  examinee  left  off 
answering  because  the  words  became  too  difficult,  or  that  he  ran 
out  of  time.  If  we  assume  the  former,  then  the  omitted  responses 
should  be  considered  icorrect  and  assigned  the  guessing 
probability  of  a  correct  response  so  as  to  be  more  equivalent  to 
non-omitted  responses  earlier  in  the  test.  If  we  assume  the 
latter,  the  omitted  responses  following  the  last  non-omitted 
responses  should  be  treated  as  not  presented. 

Considering  that  the  frequency  of  omitted  responses  at  the 
end  of  the  WK  test  is  relatively  high  (see  Table  4-8),  and 
assuming  that  the  prescribed  time  limit  had  been  adequately 
pretested,  we  have  concluded  that,  for  purposes  of  the  item 
factor  analysis,  the  omitted  items  should  be  assigned  the 
guessing  probability  of  success  for  that  item  rather  than  treated 
as  not  presented  to  that  examinee.  Scored  in  this  way,  WK  shows 
clear  evidence  of  a  second  significant  factor  (Table  4-8). 

The  interpretation  of  this  factor  is,  however,  not  at  all 
obvious.  The  principal  factor  pattern  in  Table  4-8  bears  no 
apparent  relationship  to  the  item  content,  but  resembles  instead 
the  pattern  for  a  "difficulty"  factor  encountered  when  phi 
coefficients  are  analyzed,  or  the  pattern  found  in  section  3.1 
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when  guessing  effects  were  ignored.  That  is,  the  loadings  of  the 
second  principal  factor  tend,  with  only  a  few  exceptions,  to  be 
opposite  in  sign  for  easy  and  hard  items.  Similarly,  the  promax 
factors,  which  are  highly  correlated,  divide  the  items  with 
respect  to  difficulty  or,  equivalently,  ordinal  position  in  the 
test . 

Attributing  the  significant  second  factor  to  effects  of 
difficulty  or  guessing  would  seem  to  be  ruled  out,  however,  by 
our  demonstration  in  the  simulation  study  of  section  3.1  that  the 
present  solution  is  free  of  these  artifacts.  To  eliminate  the 
possibility  that  the  solution  is  influenced  by  our  decision  to 
score  not  reached  items  as  omitted,  we  performed  an  additional 
analysis  treating  these  items  as  not  presented;  again,  a 
significant  second  factor  appeared. 

It  is  possible,  of  course,  that  in  selecting  more  difficult 
items  from  a  larger  set,  the  test  constructors  introduced  a  new 
cognitive  component  that  appears  as  a  distinct  factor.  We  have 
not,  however,  succeeded  in  identifying  any  such  component  in 
terms  of  item  features  that  vary  with  the  factor  loadings.  We 
will,  therefore,  defer  any  speculation  about  the  source  of  the 
significant  second  factor  in  the  Word  Knowledge  test  until 
evidence  for  it  can  be  found  in  other  item  sets. 

Paragraph  Comprehension  (PC)  (Table  4-9).  Only  one  factor 
was  found.  We  had  thought  that  the  several  paragraphs  on  which 
these  items  are  based  would  appear  as  factors,  but  this  was  not 
the  case.  There  is  no  evidence  of  failure  of  conditional 
independence  in  this  test.  Items  11  and  15  have  rather  poor 
discriminating  power. 
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Auto  &  Shop  Information  (AS)  {Table  4-10).  This  test, 
composed  of  Items  based  on  the  Auto  Information  and  Shop 
Information  tests  of  the  earlier  Army  Classification  Battery, 
exhibited  a  significant  and  very  clear  two-factor  pattern 
separating  the  two  types  of  items  as  already  shown  in  Figure  3-3. 
As  mentioned  in  section  3.2,  the  pattern  indicates  that  a  few  of 
the  items  are  misclassif ied .  Although  a  third  factor  could  be 
extracted  in  which  a  few  of  the  loadings  suggested  a  distinction 
between  wood-shop  and  metal-shop  items,  it  was  not  significant 
when  a  design  effect  of  2.0  was  assumed  and  is  not  reported  here. 

Mathematics  Knowledge  (MK)  (Table  4-11).  Two  factors  of 
mathematics  knowledge  are  statistically  significant;  the  third  is 
not  when  a  design  effect  of  2.0  is  assumed.  Items  with  large 
loadings  on  the  first  promax  factor  all  require  knowledge  of 
formal  algebra,  while  those  loadings  on  the  second  factor  involve 
numerical  calculation  and  mathematical  reasoning.  If  a  third 
factor  is  extracted  (not  shown),  it  tends  to  separate  calculation 
from  reasoning  but  not  clearly  so. 

Mechanical  Comprehension  (MK)  (Table  4-12).  There  is  perhaps 
marginal  evidence  of  a  second  factor  in  this  test,  but  it  is 
represented  by  only  two  items  (10  and  14).  These  items  ask  about 
the  speed  with  which  something  turns,  whereas  most  of  the  other 
items  ask  only  about  direction  of  movement  or  rotation.  Item  18, 
which  asks  about  both  direction  and  speed,  loads  on  both  factors. 
The  same  is  true  of  item  22,  but  it  loads  more  on  the  first 
factor.  The  distinction  is  of  minor  importance  at  best. 
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Electronics  Information  (El)  (Table  4-13).  This  test  shows 


no  evidence  of  a  significant  second  factor  when  a  design  effect 
of  2.0  is  assumed.  Except  for  number  14,  the  items  are  highly 
uniform  in  discriminating  power. 

4.4  DAT  Spatial  Reasoning 

In  a  study  of  item  features  requiring  spatial  visualizing 
ability,  Zimowski  (1985)  carried  out  a  full-information  item 
factor  analysis  of  the  Spatial  Visualization  subtest  of  the 
current  edition  of  the  Differential  Aptitude  Test  battery 
(Bennett,  Seashore,  and  Wesman ;  1974).  Examinees  were  390  high 
school  seniors  from  a  suburban  Chicago  school  system.  The 
analysis  revealed  four  statistically  significant  factors. 
Considering  that  the  test  consists  exclusively  of  pattern  folding 
items,  we  found  this  result  surprising.  Upon  examining 
the  items  loading  most  heavily  on  a  given  factor,  we  found  that 
they  were  based  on  basically  the  same  stimulus  pattern,  but 
modified  with  additional  marks  and  features  so  as  to  serve  as  a 
distinct  items.  Probably  the  items  were  constructed  in  this  way 
to  reduce  the  amount  of  original  drawing  required. 

That  these  factors  could  represent  distinct  cognitive 
processes  seems  unlikely.  A  more  plausible  explanation  is  that  a 
correct  response  on  the  first  encounter  with  one  of  these  similar 
sets  of  items  increases  the  probability  of  a  correct  response  to 
later  items  from  the  set,  while  an  incorrect  response  on  the 
first  encounter  does  not  lead  to  an  increase,  These  failures  of 
conditional  independence  would  produce  increased  associations 
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among  items  that  would  appear  as  a  factor.  It  may  be  possible  to 
distinguish  this  type  of  factor  from  a  genuine  cognitive  process 
factor  by  position  effects.  Positively  associated  items  should 
become  less  difficult  as  they  are  preceded  by  more  items  from  the 

same  dependent  set.  This  sort  of  violation  of  standard  item- 
response  theoretic  assumptions  could  easily  be  corrected  by 
avoiding  repeated  use  of  similar  features  among  items  in  the  same 
scale.  Unfortunately,  this  strategy  would  rule  out  scales 
consisting  of  items  generated  by  varying  components  of  a  facet 
design  on  the  item  content  or  formats.  This  finding  is  discussed 
in  greater  detail  in  Zimowski  (1985a). 


5  Discussion  and  conclusion 

Implementation  of  item  factor  analysis  by  marginal  maximum 
likelihood  estimation  overcomes  many  of  the  problems  that  attend 
factor  analysis  of  tetrachoric  correlation  coefficients:  it 
avoids  the  problem  of  indeterminate  tetrachoric  coefficients  of 
extremely  easy  or  difficult  items;  it  readily  accomodates 
effects  of  guessing,  and  omitted  or  not  reached  items;  and  it 
provides  a  likelihood  ratio  test  of  the  statistical  significance 
of  additional  factors.  Although  the  numerical  integration  used 
in  the  MML  approach  involves  heavy  computation  and  limits  the 
procedure  to  five  factors,  the  number  of  items  that  can  be 
analyzed  is  sufficiently  large  (up  to  60)  to  qualify  the  method 
for  use  in  practical  test  development. 

The  applications  of  the  procedure  reported  in  the  present 
paper  show  that,  in  moderately  large  samples  (500  to  1000  cases), 
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Table  4-9 


Item  Facilities,  Attempts,  Standard  Difficulties,  and  Factor  Loadings 

Paragraph  Comprehension 


Item 

Facility 

Attempts 

Difficulty 

Principal  Factor 

1 

1 

0.747 

1176 

-0.322 

0. 

786 

2 

0.841 

1176 

-0.832 

0 

676 

3 

0.772 

1176 

-0.486 

0. 

899 

4 

0.685 

1175 

-0.189 

0 

610 

5 

0.658 

1 1  74 

-0.247 

0. 

800 

6 

0.670 

1173 

-0.024 

0 

731 

7 

0.658 

1 1  73 

-0.162 

0. 

663 

8 

0.733 

1 1  73 

-0.422 

0. 

574 

9 

0.712 

1169 

-0.350 

0. 

747 

10 

0.478 

1166 

0.321 

0 

735 

1  1 

0.723 

1160 

-0.382 

0. 

483 

12 

0.566 

1150 

0.183 

0. 

711 

13 

0.735 

1136 

-0.402 

0. 

761 

14 

0.609 

1102 

0.008 

0 

.698 

15 

0.505 

1085 

0.283 

0. 

143 

Adding  Factor 

Chi-Square* 

D.F. 

P 

Percent 

Change 

of  Variance 

2 

1 1.586 

14 

0.640 

47.497 

♦Assumed  design  effect  =  2. 
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Table  4-8 


Item  Facilities,  Attempts,  Standard  Difficulties,  and 

Word  Knowledge 

Factor 

Loadings 

tem 

Facility 

Attempts 

Difficulty 

Principal 

1 

Factors 

2 

Promax 

1 

Factors 

2 

1 

0.914 

1176 

-1.173 

0.708 

0.158 

0.141 

0.606 

2 

0.902 

1176 

-1.118 

0.725 

0.153 

0.1  58 

0.606 

3 

0.857 

1176 

-0.828 

0.795 

0.146 

0.209 

0.628 

4 

0.870 

1176 

-0.909 

0.576 

0.223 

-0.040 

0.650 

5 

0.882 

1176 

-0.997 

0.709 

0.358 

-0.186 

0.938 

6 

0.812 

1176 

-0.497 

0.917 

0.148 

0.273 

0.693 

7 

0.834 

1176 

-0.733 

0.677 

-0.070 

0.494 

0.215 

8 

0.797 

1176 

-0.466 

0.891 

-0.094 

0.655 

0.279 

9 

0.621 

1176 

-0.040 

0.71  7 

0.077 

0.277 

0.478 

10 

0.866 

1175 

-0.889 

0.817 

0.131 

0.245 

0.615 

1  1 

0.726 

1174 

-0.302 

0.806 

0.042 

0.385 

0.462 

12 

0.787 

1 1  74 

-0.578 

0.702 

0.247 

-0.008 

0.751 

13 

0.806 

1174 

-0.420 

0.872 

0.099 

0.329 

0.588 

14 

0.678 

1173 

-0.078 

0.880 

0.234 

0.112 

0.817 

15 

0.71  7 

1171 

-0.322 

0.843 

-0.055 

0.563 

0.320 

16 

0.761 

1170 

-0.380 

0.788 

0.055 

0.354 

0.475 

17 

0.672 

1169 

-0.077 

0.931 

0.251 

0.113 

0.870 

18 

0.723 

1165 

-0.226 

0.792 

-0.175 

0.731 

0.096 

19 

0.635 

1161 

0.100 

0.781 

-0.240 

0.830 

-0.016 

20 

0.752 

1160 

-0.368 

0.831 

0.059 

0.370 

0.503 

21 

0.723 

1158 

-0.090 

0.807 

-0.152 

0.700 

0.143 

22 

0.624 

1152 

0.217 

0.934 

-0.015 

0.550 

0.430 

23 

0.560 

1146 

0.319 

0.850 

-0.276 

0.928 

-0.043 

24 

0.530 

1141 

0.672 

0.786 

-0.238 

0.830 

-0.01 1 

25 

0.547 

1132 

0.523 

0.845 

0.022 

0.439 

0.448 

26 

0.581 

1 1  21 

0.243 

0.895 

-0.033 

0.557 

0.382 

27 

0.551 

1110 

0.303 

0.760 

-0.098 

0.587 

0.209 

28 

0.657 

1098 

0.084 

0.723 

-0.143 

0.638 

0.117 

29 

0.486 

1083 

0.756 

0.808 

-0.103 

0.621 

0.224 

30 

0.51  7 

1065 

0.588 

0.732 

-0.222 

0.773 

-0.010 

31 

0.834 

1050 

-0.402 

0.845 

0.037 

0.415 

0.473 

32 

0.473 

1036 

0.862 

0.706 

-0.192 

0.710 

0.027 

33 

0.478 

1017 

0.873 

0.908 

-0.158 

0.767 

0.182 

34 

0.561 

1003 

0.348 

0.81 1 

0.038 

0.393 

0.458 

35 

0.509 

985 

0.504 

0.878 

-0.147 

0.733 

0.185 

dding 

actor 

Chi-square 

Change 

*  D.F. 

P 

Percent  of  Variance 

Factor  Correlation 

2 

111.470 

34 

0.000 

64.863 

2.650 

1  1.000 

2  0.815 

1 .00 

Assumed  design  effect  =  2. 


48 


Table  4-7 


Item  Facilities,  Attempts,  Standard  Difficulties,  and  Factor  Loadings 

Arithmetic  Reasoning 


Item 

Facility 

Attempts 

Difficulty 

Principal  Factors 

1  2 

Promax 

1 

Factors 

2 

1 

0.896 

1176 

-1.096 

0.480 

0.226 

0.042 

0.497 

2 

0.896 

1 1  76 

-1.109 

0.628 

0.448 

-0.164 

0.894 

3 

0.703 

1176 

-0.335 

0.787 

-0.118 

0.767 

0.035 

4 

0.662 

1 1  76 

-0.158 

0.842 

-0.064 

0.732 

0.138 

S 

0.606 

1176 

0.087 

0.746 

-0.021 

0.598 

0.178 

6 

0.665 

1176 

-0.222 

0.728 

-0.042 

0.614 

0.141 

7 

0.745 

1176 

-0.366 

0.521 

0.171 

0.151 

0.422 

8 

0.680 

1176 

-0.215 

0.702 

-0.074 

0.639 

0.083 

9 

0.645 

1176 

-0.126 

0.748 

-0.158 

0.795 

-0.039 

10 

0.606 

1176 

0.1  28 

0.893 

0.119 

0.508 

0.444 

1  1 

0.551 

1176 

0.067 

0.876 

-0.083 

0.785 

0.117 

1  2 

0.526 

1175 

0.219 

0.768 

-0.075 

0.692 

0.099 

1  3 

0.560 

1175 

0.321 

0.773 

0.034 

0.539 

0.275 

14 

0.501 

1175 

0.284 

0.821 

-0.209 

0.923 

-0.099 

15 

0.571 

1170 

0.151 

0.818 

-0.188 

0.891 

-0.067 

16 

0.565 

1167 

0.578 

0.839 

-0.046 

0.705 

0.165 

1  7 

0.478 

1167 

0.774 

0.849 

-0.163 

0.879 

-0.019 

18 

0.459 

1166 

0.886 

0.908 

0.038 

0.636 

0.319 

19 

0.493 

1164 

0.449 

0.722 

0.022 

0.518 

0.240 

20 

0.308 

1162 

0.789 

0.789 

-0.003 

0.604 

0.220 

21 

0.386 

1  1  59 

0.841 

0.880 

0.004 

0.663 

0.257 

22 

0.485 

1  1  51 

0.640 

0.880 

-0.110 

0.826 

0.076 

23 

0.481 

1145 

0.616 

0.751 

0.441 

-0.061 

0.918 

24 

0.424 

1140 

0.763 

0.871 

0.226 

0.339 

0.608 

25 

0.408 

1135 

0.81  2 

0.878 

-0.007 

0.677 

0.239 

26 

0.407 

1121 

0.621 

0.744 

-0.034 

0.61  5 

0.157 

27 

0.337 

1107 

0.705 

0.793 

-0. 144 

0.809 

-0.004 

28 

0.291 

1074 

1  .1  22 

0.868 

0.092 

0.527 

0.394 

29 

0.277 

1049 

1.148 

0.820 

0.073 

0.519 

0.350 

30 

0.392 

1018 

0.816 

0.802 

-0.1  18 

0.779 

0.040 

Adding 

Factor 

2 

3 

Chi-Square*  D.F.  P 

Change 

93.519  29  0.000 

27.525  28  0.490 

Percent  of 

62.469 

Variance 

2.587 

Factor 

Correlations 

1  1.000 

2  0.787  1.000 

♦Assumed  design  effect  =  2. 


47 


Table  4-6 


Item  Facilities,  Attempts,  Standard  Difficulties,  and  Factor  Loadings 

General  Science 


Item 

Facility 

Attempts 

Difficulty 

Principal  Factors 

1  2 

Promax 

1 

Factors 

2 

1 

0.843 

1177 

-0.827 

0.710 

-0.319 

0.008 

0.773 

2 

0.758 

1177 

-0.463 

0.737 

0.092 

0.581 

0.201 

3 

0.726 

1176 

-0.327 

0.794 

0.105 

0.633 

0.209 

4 

0.669 

1176 

-0.141 

0.626 

0.153 

0.595 

0.064 

5 

0.722 

1176 

-0.392 

0.628 

0.141 

0.580 

0.083 

6 

0.765 

1176 

-0.51 3 

0.779 

-0.264 

0.1  26 

0.724 

7 

0.672 

1176 

-0.024 

0.675 

-0.235 

0.101 

0.637 

8 

0.805 

1176 

-0.678 

0.548 

-0.234 

0.023 

0.579 

9 

0.726 

1176 

-0.354 

0.71 1 

0.093 

0.566 

0.188 

10 

0.709 

1176 

-0.322 

0.590 

0.157 

0.578 

0.043 

1 1 

0.662 

1 1  76 

-0.120 

0.715 

-0.036 

0.394 

0.374 

12 

0.513 

1176 

0.835 

0.719 

0.319 

0.876 

-0.1 28 

13 

0.472 

1175 

0.633 

0.884 

0.242 

0.875 

0.055 

14 

0.608 

1174 

0.01 1 

0.620 

0.069 

0.478 

0.181 

15 

0.685 

1171 

-0.164 

0.542 

-0.138 

0.149 

0.440 

16 

0.638 

1167 

-0.139 

0.609 

-0.294 

-0.021 

0.691 

17 

0.618 

1163 

0.052 

0.566 

-0.484 

-0.304 

0.941 

18 

0.384 

1 1  55 

0.795 

0.900 

0.045 

0.618 

0.342 

19 

0.473 

1150 

0.778 

0.765 

-0.102 

0.336 

0.489 

20 

0.477 

1142 

0.390 

0.628 

-0.095 

0.261 

0.417 

21 

0.353 

1131 

0.844 

0.651 

0.176 

0.642 

0.043 

22 

0.343 

1125 

1  .1  21 

0.799 

-0.087 

0.377 

0.484 

23 

0.338 

1104 

1.113 

0.701 

0.389 

0.960 

-0.235 

24 

0.215 

1091 

1  .365 

0.891 

0.034 

0.598 

0.353 

25 

0.358 

1055 

1 .270 

0.933 

0.044 

0.637 

0.358 

Adding 

Factor 

2 

3 

Chi-Square*  D.F 

Change 

67.227  24 

14.181  23 

P 

0.000 

0.922 

Percent  of 

51 .457 

Variance 

4.391 

Factor 

Correlations 

1  1.000 

2  0.740  1.000 

ASVAB  8A  Guessing  Parameter  Values 
from  Bock  and  Mislevy  (1981) 


Quality  of  Life  Data: 

Analysis  by  Marginal  Maximum  Likelihood 


Hey wood  case 


Table  4-1 


Tetrachoric  Correlation  Coefficients  of  the  LSAT-7  Items 
(Coefficients  Corrected  for  Guessing  above  the  Diagonal:  g=0.2) 

(N=1000) 


1 

2 

Item 

3 

4 

5 

1 

1  .000 

0.294 

0.358 

0.401 

0.344 

2 

0.226 

1.000 

0.567 

0.288 

0.174 

Item  3 

0.291 

0.432 

1.000 

0.376 

0.325 

4 

0.296 

0.204 

0.277 

1  .000 

0.214 

5 

0.286 

0.135 

0.265 

0.161 

1 .000 

Table  4-2 

Chi 

-square  Statistics  for  the  Two-Factor  Stepwise 
Analysis  With  and  Without  Guessing:  LSAT-7 
(N=1000) 

No  Guessing 
Chi-square  D.F. 

P 

Guessing 

Chi-square  D.F. 

P 

One-Factor 

31.66 

21 

0.063 

32.94 

21 

0.047 

Two-Factor 

22.86 

17 

0.154 

24.80 

17 

0.099 

Change 

8.80 

4 

0.066 

8.14 

4 

0.086 

Table  4-3 

LSAT-7  Residual  Correlations 
(Guessing  above  Diagonal) 

1 

2 

Item 

3 

4 

5 

1 

— 

0.016 

-0.005 

0.043 

0.032 

2 

0.009 

— 

0.000 

0.005 

-0.048 

Item  3 

-0.024 

0.003 

— 

0.037 

0.050 

4 

0.026 

-0.003 

0.018 

— 

-0.036 

5 

0.017 

-0.015 

0.034 

-0.042 

— 

4 


.  ■. 
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Table  3-2 


Change  of  the  Likelihood  Ratio  Chi-square  upon  Adding  a 
Second  Factor  to  the  Models  With  and  Without  Guessing 
Analysis  of  Unidimensional  Simulated  Data 


Model 

Chi-square 

d.f . 

P 

No  Guessing 

39.166 

20 

0.006 

Guessing 

26.928 

20 

0.137 

Table  3-3 

Change  of  the  Likelihood  Ratio  Chi-square  in  the  Factor 
Analysis  of  the  Auto  and  Shop  Information  Test 


Factor 

Chi-square* 

d.f. 

P 

2  vs.  1 

175.6 

24 

0.000 

3  vs.  2 

24.7 

23 

0.363 

•Assumed 

design  effect  *  2. 

Table  3-1 


Principal  Factor  Loadings  from  Simulated  Data  With  Guessing 
Effect  Analyzed  by  No-guessing  and  Guessing  Models* 


Non-Guessing  Model  Guessing  Model 


Item 

Principal 

1 

Factors 

2 

Principal  Factor 

1 

1 

0.703 

0.147 

0.761 

2 

0.719 

0.046 

0.724 

3 

0.739 

0.215 

0.732 

4 

0.654 

-0.029 

0.684 

5 

0.642 

0.069 

0.660 

6 

0.689 

0.124 

0.736 

7 

0.660 

0.065 

0.697 

8 

0.704 

0.129 

0.755 

9 

0.580 

-0.032 

0.697 

10 

0.561 

-0.106 

0.697 

1  1 

0.574 

-0.049 

0.710 

12 

0.583 

-0.204 

0.765 

13 

0.505 

-0.102 

0.715 

14 

0.393 

-0.213 

0.665 

15 

0.407 

-0.168 

0.704 

16 

0.329 

0.003 

0.716 

17 

0.274 

-0.068 

0.688 

18 

0.21 1 

-0.081 

0.653 

19 

0.148 

-0.545 

0.724 

20 

0.041 

-0.068 

0.594 

21 


0.128 


0.069 


0.759 


Table  2-1 


Original  Proportions  of  Subjects  Passing 
and  Failing  Items  i  and  j 


Pass 

Item  j 
Fail 

Total 

Pass  TT^ 

*10 

"1. 

Item  i 

Fail  it  o  i 

O 

o 

Total  ir  j 

\0 

1  .0 

Table  2-2 

Corrected  Proportions  of 
and  Failing  Items 

Subjects 
i  and  j 

Passing 

Pass 

Item  j 
Fail 

Total 

Pass  ir'  1 1 

*'  10 

Item  i 

Fail  ir'01 

»'  00 

*’0. 

Total  ir'  j 

*'.0 

1  .0 

Table  2-3 

Observed  Frequencies  of  Subjects  Passing, 
Failing,  and  Omitting  Items  i  and  j 

r 


Pass 


Item  j 
Fail 


Omit 


Total 
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minor  factors  determined  by  relatively  few  items  can  be  detected 
as  signif  icars*- .  The  sensitivity  of  the  MML  method  recommends  it 
as  an  exploratory  technique  in  searching  for  item  features  that 
are  responsible  for  individual  differences  in  cognitive  test 
performance.  By  the  same  token,  format  attributes  that  may  be 
implicated  in  failures  of  conditional  independence  are  easily 
detected . 

The  examples  presented  in  section  4.3  suggest  that  many 
routinely  used  tests  may  contain  some  items  that  produce 
departures  from  unidimensionality  or  conditional  independence. 

In  many  situations  such  items  could  be  eliminated  by  including 
in  the  same  scale  only  items  that  are  highly  homogeneous  in  all 
content  and  format  features  that  are  not  relevant  to  the  ability 
dimension  in  questions.  Otherwise,  the  only  practical 
alternative  may  be  to  integrate  over  the  distributions  of  ability 
in  these  minor  dimensions  when  estimating  the  posterior  mean  for 
the  main  dimension,  given  the  examinee's  item  response  vector. 
This  is  effectively  what  is  occuring  when  a  single  score  is 
reported  for  a  test  in  which  the  items  are  not  strictly 


unidimensional . 


Item 


Facility  Attempts  Difficulty  Principal  Factors  Promax  Factors 

12  12 


1 

0.704 

1176 

-0.300 

0.381 

0.201 

-0.058 

0.471 

2 

0.768 

1176 

-0.565 

0.592 

-0.026 

0.368 

0.268 

3 

0.602 

1176 

0.015 

0.753 

-0.295 

0.822 

-0.019 

4 

0.799 

1176 

-0.651 

0.604 

0.079 

0.233 

0.417 

5 

0.615 

1176 

-0.038 

0.849 

-0.117 

0.635 

0.275 

6 

0.491 

1176 

0.263 

0.876 

-0.132 

0.671 

0.268 

7 

0.467 

1176 

0.532 

0.818 

0.025 

0.426 

0.454 

8 

0.603 

1176 

-0.037 

0.465 

0.168 

0.034 

0.469 

9 

0.633 

1176 

-0.104 

0.356 

0.200 

-0.070 

0.457 

10 

0.545 

1176 

0.188 

0.762 

0.248 

0.093 

0.730 

1  1 

0.551 

1175 

0.265 

0.584 

0.339 

-0.130 

0.764 

12 

0.556 

1174 

0.093 

0.469 

0.242 

-0.063 

0.572 

1  3 

0.558 

1174 

0.006 

0.701 

-0.210 

0.678 

0.071 

14 

0.582 

1174 

0.131 

0.779 

0.127 

0.267 

0.573 

15 

0.469 

1171 

0.390 

0.769 

-0.137 

0.617 

0.206 

16 

0.467 

1166 

0.412 

0.806 

0.081 

0.344 

0.524 

17 

0.379 

1161 

0.710 

0.895 

-0.105 

0.644 

0.316 

18 

0.383 

1157 

0.791 

0.930 

-0.137 

0.708 

0.289 

19 

0.593 

1154 

-0.092 

0.545 

0.138 

0.1  20 

0.469 

20 

0.477 

1147 

0.447 

0.666 

-0.149 

0.576 

0.137 

21 

0.379 

1132 

0.875 

0.655 

0.1  23 

0.202 

0.505 

22 

0.379 

1126 

0.697 

0.870 

-0.237 

0.809 

0.121 

23 

0.262 

1114 

0.802 

0.906 

-0.143 

0.703 

0.268 

24 

0.273 

1093 

1 .086 

0.841 

0.111 

0.323 

0.583 

25 

0.371 

1075 

0.780 

0.536 

0.286 

-0.085 

0.667 

Adding 

Chi-Square* 

D.F. 

P 

Percent  of 

Variance 

Factor 

Factor 

Change 

Correlations 

2 

75.572 

24 

0.000 

51.272 

3.243 

1  1.000 

2  0.731  1.00 

Item  Facility  Attempts  Difficulty 


Principal  Factors  Promax  Factors 


1 

2 

1 

2 

1 

0.803 

1175 

-0.647 

0.780 

-0.376 

-0.230 

1.054 

2 

0.721 

1174 

-0.395 

0.620 

-0.163 

0.068 

0.582 

3 

0.535 

1 1  74 

0.108 

0.768 

-0.081 

0.306 

0.494 

4 

0.652 

1174 

0.067 

0.845 

-0.023 

0.460 

0.418 

5 

0.680 

1174 

-0.262 

0.576 

-0.128 

0.106 

0.497 

6 

0.519 

1173 

0.252 

0.843 

0.013 

0.524 

0.350 

7 

0.608 

1173 

0.175 

0.922 

0.151 

0.824 

0.126 

8 

0.523 

1173 

0.177 

0.684 

-0.007 

0.393 

0.317 

9 

0.598 

1173 

0.242 

0.836 

0.131 

0.736 

0.125 

10 

0.561 

1173 

0.202 

0.746 

0.006 

0.454 

0.320 

1  1 

0.509 

1171 

0.594 

0.780 

0.078 

0.606 

0.200 

1  2 

0.422 

1 1  70 

0.475 

0.839 

-0.179 

0.168 

0.710 

13 

0.469 

1168 

0.457 

0.945 

0.024 

0.605 

0.374 

14 

0.386 

0.646 

0.907 

-0.046 

0.452 

0.490 

15 

0.388 

1163 

0.931 

0.597 

-0.051 

0.259 

0.362 

16 

0.493 

1159 

0.676 

0.946 

0.080 

0.708 

0.269 

1  7 

0.379 

1158 

0.617 

0.889 

0.111 

0.732 

0.185 

18 

0.431 

1158 

0.624 

0.934 

0.122 

0.784 

0.190 

19 

0.502 

1157 

0.671 

0.834 

-0.253 

0.029 

0.846 

20 

0.419 

1152 

0.821 

0.854 

0.065 

0.627 

0.257 

21 

1147 

1.115 

0.884 

0.199 

0.891 

0.018 

22 

0.318 

1143 

1  .064 

0.940 

0.147 

0.828 

0.141 

23 

0.269 

1135 

1  .083 

0.905 

-0.067 

0.414 

0.527 

24 

0.264 

1114 

1.055 

0.778 

-0.116 

0.247 

0.565 

25 

0.281 

1084 

1 .073 

0.923 

0.144 

0.813 

0.139 

Adding  Chi-Square*  D.F.  P 

Factor  Change 


Percent  of  Variance 


Factor 

Correlations 


2  77.633 

3  27.998 


24  0.000 

23  0.216 


68.954  1.903  1 

2 


1  .000 

0.856  1.000 


Item 

Facility 

Attempts 

Difficulty 

Principal  Factors 

1  2 

Promax 

1 

Factors 

2 

1 

0.865 

1175 

-0.948 

0.496 

0.032 

0.378 

0.144 

2 

0.727 

1175 

-0.373 

0.744 

-0.076 

0.729 

0.024 

3 

0.740 

1175 

-0.464 

0.587 

-0.015 

0.516 

0.088 

4 

0.380 

1175 

0.757 

0.748 

0.028 

0.597 

0.186 

5 

0.543 

1175 

1 .079 

0.692 

-0.118 

0.740 

-0.052 

6 

0.580 

1174 

0.860 

0.766 

-0.017 

0.671 

0.120 

7 

0.518 

1174 

0.622 

0.864 

-0.01 1 

0.746 

0.147 

8 

0.557 

1174 

0.042 

0.792 

0.010 

0.658 

0.166 

9 

0.617 

1174 

-0.081 

0.519 

-0.151 

0.636 

-0.134 

10 

0.530 

1174 

0.096 

0.832 

0.237 

0.395 

0.524 

1  1 

0.609 

1174 

0.030 

0.427 

-0.077 

0.462 

-0.038 

12 

0.512 

1174 

0.418 

0.814 

-0.078 

0.791 

0.034 

0.598 

1174 

0.196 

0.779 

0.072 

0.754 

0.037 

14 

0.518 

1173 

0.258 

0.753 

0.607 

-0.155 

1  .081 

15 

0.498 

1173 

0.237 

0.712 

0.031 

0.562 

0.184 

16 

0.541 

1170 

0.157 

0.555 

-0.145 

0.660 

-0.118 

17 

0.472 

1168 

0.827 

0.800 

-0.212 

0.954 

-0.175 

18 

0.446 

1163 

0.702 

0.868 

0.182 

0.498 

0.445 

19 

0.436 

1157 

1.292 

0.847 

-0.003 

0.722 

0.156 

20 

0.474 

1146 

0.461 

0.639 

-0.042 

0.596 

0.057 

21 

0.397 

1 1  38 

0.905 

0.830 

-0.169 

0.923 

-0.103 

22 

0.381 

1124 

0.107 

0.786 

0.068 

0.577 

0.254 

23 

0.330 

0.750 

0.725 

-0.010 

0.627 

0.123 

24 

0.386 

1078 

0.718 

0.686 

-0.057 

0.664 

0.044 

25 

0.327 

1062 

0.891 

0.797 

-0.083 

0.783 

0.024 

Adding 

Factor 

2 

3 

Chi-Square 

Change 

29.982 

1 5.933 

*  D.F, 

24 

23 

.  P 

0.185 

0.858 

Percent  of  Variance 

53.643  2.527 

Factor 

Correlations 

1  1.000 

2  0.766  1.000 

•Assumed  design  effect  =  2. 
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Table  4-13 


Item  Facilities,  Standard  Dif ficulities,  and  Factor  Loadings 
Electronics  Information 


Item 

Facility 

Attempts 

Difficulty 

Principal  Factors 

1 

1 

0.757 

1176 

-0.512 

0.619 

2 

0.674 

1176 

0.056 

0.846 

3 

0.639 

1176 

-0.102 

0.761 

4 

0.662 

1176 

-0.236 

0.761 

5 

0.703 

1176 

-0.305 

0.607 

6 

0.625 

1176 

-0.093 

0.764 

7 

0.636 

1175 

-0.007 

0.699 

8 

0.605 

1174 

-0.053 

0.676 

9 

0.652 

1173 

-0.061 

0.564 

10 

0.496 

1173 

0.194 

0.682 

1  1 

0.415 

1171 

0.910 

0.628 

1  2 

0.420 

1169 

0.598 

0.814 

0.376 

1164 

0.624 

0.724 

14 

0.458 

1161 

0.437 

0.387 

15 

0.403 

1157 

0.564 

0.805 

16 

0.394 

1150 

0.692 

0.670 

1  7 

0.252 

1138 

1  .920 

0.704 

18 

0.389 

1131 

0.532 

0.611 

19 

0.405 

1115 

0.689 

0.731 

20 

0.289 

1101 

1  .1  28 

0.780 

Adding 

Chi-Square* 

D.F. 

P 

Percent  of  Variance 

Factor  Change 


2 


21.773 


19 


0.296 


48.879 


Average  Tetrachoric  Correlation 


Set  of  Items  Ordered  by  Facility 


0.0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9  1.0 


Figure  3-3 


Factor  loadings  for  observed  and  simulated  Auto  & 
Information  Test 


Shop 


56 


V  v, 


.•  >>>y  >>: 


Marginal  Log  Likelihood 


Figure  4-1  Increase  in  Marginal  Log  Likelihood  in 
of  a  Two  Factor  Solution  without  Guess 
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